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Abstract 

(FULL PAPER IN FRONTIERS IN NEURAL CIRCUITS, 2010) We show that the local Spike 
Timing-Dependent Plasticity (STDP) rule has the effect of regulating the trans-synaptic weights 
of loops of any length within a simulated network of neurons. We show that depending on STDP's 
polarity, functional loops are formed or eliminated in networks driven to normal spiking conditions 
by random, partially correlated inputs, where functional loops comprise weights that exceed a non- 
zero threshold. We further prove that STDP is a form of loop-regulating plasticity for the case of a 
linear network comprising random weights drawn from certain distributions. Thus a notable local 
synaptic learning rule makes a specific prediction about synapses in the brain in which standard 
STDP is present: that under normal spiking conditions, they should participate in predominantly 
feed- forward connections at all scales. Our model implies that any deviations from this prediction 
would require a substantial modification to the hypothesized role for standard STDP. Given its 
widespread occurrence in the brain, we predict that STDP could also regulate long range synaptic 
loops among individual neurons across all brain scales, up to, and including, the scale of global 
brain network topology. 



I. INTRODUCTION 



Connections between individual neurons in the brain are constrained first by the spa- 
tial distribution of axons and dendrites within the neuropil [J][2]- Global brain networks 
comprise dense connections within tissues, the gross structures in which these tissues are 
embedded, and the bidirectional long-range projections joining these structures. The topol- 
ogy of these networks is not yet fully specified at the level of microcircuitry, however [24J . 
One theoretical constraint on this level of organization, the "no strong loops hypothesis," 
considered only developmentally determined area to area connectivity patterns to imple- 
ment its specific neuron to neuron network topological constraint [3]- While local synaptic 
modifications are known to directly shape the pattern of connectivity in local neural tissue 
and thus local microcircuit topology [I] , our understanding of global brain network topology 
still derives largely from this developmentally patterned, area to area connectivity Further- 
more, measuring simultaneously the relative strengths of specific microcircuit connections 
remains technically challenging, and virtually impossible for even medium sized (100 — 200 
neurons, 0.05 — 0.1 mm) microcircuits. For these reasons, it is not yet known how large 
scale, long range microcircuit topology and the computation it supports emerges through 
synaptic modifications in the brain. 

We wondered whether a synaptic modification commonly observed in local circuit prepa- 
rations and widely hypothesized to shape local dynamics in brain structures, STDP [3], 
could be analyzed to yield an understanding of what topology it predicts for microcircuits 
of any scale. The STDP model is a departure from traditional Hebbian models of learning, 
which state that neurons that fire action potentials together will have their interconnections 
strengthened. Instead, STDP takes into account the particular temporal order of pre- and 
post-synaptic neuronal firing [6], such that the rule modifies synapses anti-symmetrically, 
depending on whether the pre- or post-synaptic neuron fires first (Fig. 1). The basic ques- 
tion we then aimed to answer is: what is the influence of this anti-symmetry on brain 
microcircuit topology? 

Consider first if a pre-synaptic "trigger" neuron causes a post-synaptic, first-order "fol- 
lower" neuron to fire. If this follower makes a direct feedback connection onto the trigger, the 
feedback connection will be weakened, since the spike generated by the follower will arrive 
at the follower-trigger synapse immediately after the trigger neuron's backward propagating 
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action potential (Fig. 1). The principle that STDP is suitable for eliminating strong recur- 
rent connections between two neurons was originally proposed by Abbott and Nelson [7]. 
Here we expand on the principle with the observation that it holds for all polysynaptic loops 
connecting triggers and followers: if some n -order follower's action potential produces in 
the original trigger a subthreshold potential after the trigger has fired, the functional loop 
will be broken by spike-timing dependent synaptic weakening of the feedback connection. 
With this intuition, we set out to prove analytically and by means of numerical simulation 
that network topology, and specifically the occurrence of functional loops in highly connected 
networks, is directly and necessarily regulated by STDP. 

This theory paper provides clear predictions about STDP's effect on neural circuit topol- 
ogy. The proof and simulations dictate strong constraints on local and long range micro- 
circuit connectivity. We propose that if these constraints are not obeyed by real neural 
circuits, the hypothesis that standard STDP shapes the structure and function of real ner- 
vous systems must be revised. Our approach suggests that similar analyses of other learning 
rules may impose similar constraints on neural circuit topology and that the hypothetical 
significance of these rules may similarly be tested. 

II. RESULTS 

A. A proof of STDP as a form of loop-regulating plasticity 

First, we represent STDP acting on a weight w associated with the connection between 
two neurons and their output variables x(t) to y(t), in the adiabatic approximation (i.e. 
small learning rate), as: 

/oo 
C xy (t)S(t)dt (1) 
-oo 

where C xy {t) = J x(if —t)y(f) is the correlator, and S(t) is the anti-symmetric STDP update 
function, S(t < 0) = exp(At), S(t > 0) = — exp(— \t). Consider this function operating over 
connections within a linear network driven by uncorrelated Gaussian inputs, £, such that 
x(t) = Wx(t) + where a; is a vector of activities with components denoted by Xi, the 
weight connection matrix has components Wy, and the input satisfies + t))* = 

a 2 S(r)Sij. We show (see Appendix 
an update for the network weight matrix of the form AW = AW(W, r, Cq) where r is the 



A 1 ) that the learning rule defined in Eq. b] results in 
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time constant of the STDP's exponential, and Co is the instantaneous correlator C(0). This 
update rule influences global network topology in a very specific way. 

To formalize our original intuition analytically, consider a linear network with only ex- 
citatory connections, such that the dynamics may be expressed as x = Wx = (— I + A)x, 
where Ay > is the network connectivity matrix (comprising the off-diagonal elements of 
the weight matrix and zeros on the diagonal), and —I represents a self-decay term. Next, 
we introduce a "loopiness" measure that estimates the strength of all loops of all sizes that 
occur in the network, £\ = Ylk=i f tr [A h ] ■ The function tr [•] stands for the trace operation; 
this operation, when acting on the fc-exponentiation of the adjacency matrix (comprising 
ones and zeros, where the nonzero entry ay represents a connection from network node i 
to network node j), counts the total number of closed paths of length exactly equal to k, 
i.e. fc-loops [25J. When applied to the network connectivity matrix A, the operation counts 
loops weighted by the product of the synaptic strengths of the looping connections, resulting 
in a slightly different, but still useful, measure of loopiness. 

It is possible, however, to reduce this measure without actually regulating topology by 
simply reducing the weights of all connections. A topological loopiness measure should 
therefore include a penalty to the weights' vanishing; we choose — |tr [AA T ], ( T stands for 
the transpose operation) which, for weighted graphs, measures the sum of the squares of all 
network weights (and for a binary graph, counts the number of links). We then define the 
total topological loopiness as: 



oo 

£ = E^ tr iw k ] - h* l AAT ] ( 2 ) 

k=l 

We showed computationally that for every weight matrix examined, each drawn from certain 



random distributions (see Appendix Al), the change in this energy as a function of the 
evolution of the network under STDP, AS ~ tr [<9a£AA t ] , is strictly semi-negative, and 
therefore STDP necessarily regulates this measure, resulting in a decrease in topological 
loopiness. We therefore use the term STDP and "loop-regulating plasticity" interchangeably 
throughout. 
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B. Loop-regulating plasticity in a network of simulated neurons 



What are the effects of this form of plasticity on network topology (and specifically 
on the number of functional loops) in nonlinear networks, such as those found in neural 
microcircuits? Because our proof of STDP as a form of loop-regulating plasticity applies 
only to linear networks or nonlinear networks that may be linearized, we aimed to show, 
using simulation, that the same principle extends to a biologically relevant, nonlinear regime. 



We replicated the simulation of Song and Abbott [8], extending it in three ways (see IV A). 
First, we created a network of 100 neurons, each receiving excitatory synapses from all 
other 99 "intra-network" input sources and from 401 randomly spiking "extra-network" 
input sources selected at random from 2,500 homogeneous Poisson processes. All excitatory 
synapses underwent STDP. Second, we provided 250 inhibitory synapses to each neuron, 
sampled from 1,250 spiking sources; the inhibitory inputs modeled fast local inhibition to the 
network using inhomogeneous Poisson processes with rates modulated by the instantaneous 
aggregate firing rate of the network. Third, we explored four different forms of STDP update 
[9] and observed robust loop-regulating plasticity for each; the results presented here used 
the STDP update rule of Giitig et al. [10J. 

We initialized our network with maximum extra-network weights, and intra-network 
weights at half maximum. This caused the network to spike vigorously when extra-network 
inputs became active, but spike rates were limited by the fast local-inhibition. After 20 sec- 
onds of simulated network activity, STDP had a profound effect on topological loopiness as 
defined in Eq. [2j measured over loops of length 2 < k < 100 for convenience (Fig. 2A). We 
counted the number of closed, functional loops of varying length using tr [ \A~\ fc ] , where \A\ 
was constructed by applying a sliding threshold to the network connectivity matrix (Fig. 
2B). We compared this quantity to the same, measured for a randomized network, con- 
structed by randomly reassigning weights from the learned weight distribution to synapses 



in the network (see IV B). These results are representative of all loop lengths measured 
(2 < n < 100) and show that as the weight threshold grows, the number of closed, func- 
tional loops in the STDP-learned network decreases more than in the randomized network. 
This form of loop-regulating plasticity can therefore be described as loop-eliminating. 
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C. The effect of synaptic delays on loop-regulating plasticity 



We wondered what effect synaptic delays would have on this result, since we expected 
follower feedback spikes to cause less anti-loop learning as they fell further from the zero 
time difference maxima in the STDP update function. We also wondered if the decrease in 
the number of functional loops compared to a randomized network also applied to unique 
loops, in which no neuron is traversed more than once. We therefore sampled the number 
of unique, functional loops through networks simulated with synaptic delays from 0.1 to 4.0 
milliseconds. We constructed one million random paths of length k — 1 for each loop length 



2 < k < 25, and for the learned and randomized networks (see IV B). We searched for each 
path across all networks studied, and if the path and the k th link completing the functional 



loop existed in the network, we counted it for that network (see IV B). The result is similar 
to that for closed loops, and, as expected, longer synaptic delays resulted in an exponential 
decrease in the number of loops as a function of loop length that deviated less from the same 
function for randomized networks, indicating weaker loop-regulating plasticity (Fig. 2C). 



D. Network in-hubs, out-hubs, and loop-regulating plasticity 

Next, we asked if other topological measures of the STDP-learned networks may be cor- 
related with our observation of STDP's effect on loopiness, since many different topological 
properties might coincide with or support this effect. For example, one means to create 
networks poor in loops is to ensure that nodes in the network are either "out-hubs" or "in- 
hubs," but not both [11]. An out-hub in a network of neurons has many strong postsynaptic 
connections but few strong presynaptic connections, and an in-hub has many strong presy- 
naptic connections but few strong postsynaptic connections. We applied a sliding threshold 
to the network connectivity matrix learned by STDP, and examined the manifold, colored 
according to each applied threshold, which correlated in-degree versus out-degree for each 
neuron in our network. This showed a clear inverse relationship between in- and out-degrees 
that varied in form with weight threshold (Fig. 3A). In contrast, by examining the in- 
degree from extra- network inputs, we found a positive correlation (Fig. 3B), indicating that 
out-hubs were more likely to be in-hubs within the larger extra-network topology, and that 
in-hubs in our network were more likely to receive only the weakest extra-network inputs. 
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E. Reverse STDP restored loops after loop-eliminating plasticity 



Beyond these standard topological analyses, we also examined biological properties of the 
network. We measured total synaptic input as a function of total synaptic output for all 
neurons in the STDP-learned network. In the same experiment, we asked if reversing the 
polarity of the standard STDP function might undo the effects of loop-regulating plastic- 
ity that results from standard STDP, since under this "reverse" condition, follower spikes 
would cause strengthening of closed-loop feedback connections. This reversal of polarity is 
biologically relevant, since it occurs at the synaptic interface between major brain struc- 
tures such as neocortex and striatum [T2], arises specifically at synapses between certain 
cell types, and is controlled by cholinergic and adrenergic neuromodulation, for example in 
the neocortical microcircuit |13j . We found the same inverse relationship between in-degree 
versus out-degree for each neuron in our network (Fig. 4A, green markers), as well as an in- 
verse relationship between total synaptic input and output following 1.5 seconds of standard 
STDP (Fig. 4B, green markers). These effects contributed to a reduction in the number 
of closed loops (Fig. 4C, depicted as in Fig. 2B), and each of these relationships could 
be largely abolished by 3 to 5 additional seconds of reverse STDP (Fig. 4, red markers), 
in contrast to 3 to 5 additional seconds of standard STDP (Fig. 4, blue markers), which 
strengthened them. We also found the same positive correlation between in-degree from 
extra-network inputs and out-degree within the network (Fig. 4D) and between total extra- 
network synaptic input and total intra-network synaptic output (Fig. 4E). This effect was 
also largely abolished by 3 to 5 seconds of reverse loop-regulating plasticity, but reinforced 
by 3 to 5 seconds of standard loop-regulating plasticity. 

F. Dynamical effects of loop-regulating plasticity 

What are the consequences of this form of network plasticity beyond topology? In the case 
of a linear network, reducing the number of loops implies more stable dynamics. Consider 
the stability of the unforced system x(t) — Wx(t) = 0; the eigenvalues A of W = —I + A 
can be expressed as: 




(3) 



i=i 



k=l 
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which emphasizes the contribution of loops to system instability. Such a simple ob- 
servation, however, does not make clear predictions about the effects of loop-regulating 
plasticity on nonlinear neural circuit function. We were surprised to find that raster plots 
of network spiking activity, when sorted according to certain topological metrics (e.g., the 
sum of extra-network input weights, the sum of intra-network output weights, in-degrees, 
out-degree) consistently revealed network events that originate with weak synchronization 
among out-hubs, followed by strong synchronization among in-hubs (Fig. 5A, top), across 
8 independent simulations of the phenomenon. This effect was altered by randomizing the 
intra-network weights, such that synchronization events became stronger, more frequently 
global, and more frequent among out-hubs alone (Fig. 5A, bottom). Peri-event time his- 
tograms constructed across 8 independent simulations reveal this same effect (Fig. 5B, left 
panels), with synchronization arising strongly among in-hubs after weak out-hub activation 



in the STDP-learned network, and globally in the randomized network (see IV B for a de- 
scription of how network events were detected). In the STDP-learned network, both in-hubs 
and out-hubs sustain spike rates ranging from 4 — 9 Hz that are not correlated with in-degree, 
whereas in the randomized network, spike rates range more broadly (3 — 16 Hz) and are 
highly correlated with in-degree (Fig. 5B, right panels). We examined the summed network 
peri-event time histograms for the STDP-learned network and for networks that underwent 
randomization of their intra-network weights, their extra-network weights, or both (Fig. 
5C, top), across 8 simulations for each condition. The resulting distributions, as well as 
a pooled distribution of times from all randomized networks each differed from each other 
based on paired Kolmogorov-Smirnov tests (P w 0). To examine what properties of these 
distributions distinguished them, we measured kurtosis and skew for each distribution from 
each simulation, and compared the distributions of kurtosis and skew measures between 
each group. Kurtosis differed significantly between the STDP-learned network topology 
and the randomized topologies (intra-network randomized, extra-network randomized, both 
randomized, Fig. 5C, bottom). Results from unpaired t-tests of the distribution of skew 
and kurtosis measurements between 4 simulation conditions (calculated separately for each 
of the 8 simulations) are shown in table inset (Fig. 5C, bottom). These effects indicate 
that the effect of standard loop-regulating plasticity is to generate network topologies that 
support network events with greater spread and sharper peaks in time. 
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III. DISCUSSION 



Based on our simulations and analytical results, we propose that standard STDP must 
produce a network topology in real neural tissues that is conspicuously poor in both closed 
and unique loops, and that it will segregate neurons into out- and in-hubs to achieve this. 
Such a prediction can easily be tested by analyzing correlations between the number of 
functional input connections and the number of functional output connections made by 
neurons recorded during multi-patch clamp experiments in a structure in which STDP has 
been observed (e.g., [I], [IS])- Our theory predicts this correlation should be negative. 

The network that emerges in such tissues will organize its relationship to inputs from 
other structures in an orderly fashion, making local out-hubs the primary target for long 
range inputs, and thus establishing a feed forward relationship between the network and its 
pool of inputs. In a larger system, we anticipate that local in-hubs would become long range 
outputs. This prediction may also be tested by correlating the local topological relationships 
of a neuron with its identified role as either an input, output, or interneuron within that 
structure. 

We also make a clear prediction for the effect of synaptic delays on modulating the 
topological effects of STDP. Correlations between these delays and functional connectivity 
data from multi-patch clamp recordings from connected neurons undergoing STDP can also 
be measured to determine if synaptic delays predict the strength of reciprocal connections. 
Furthermore, we observe that, at interfaces between brain structures where STDP is reversed 
by neuromodulation, circuit dynamics can be predicted from the expected change in network 
topology. For example, changes in STDP at the cortico-striatal synapse [H] resulting in 
reverse STDP would favor the emergence of strong cortico-striatal-thalamocortical loops 
resulting in oscillations in this circuit. 

Interestingly, the depletion of loops and the separation of nodes into out-hubs and in-hubs 
has been recently reported in a variety of complex biological systems, including functional 
networks at the level of spatio-temporal resolution of fMRI, and the neural network of 
C. Elegans [11] . suggesting a general principle of organization and dynamical stability for 
entire classes of functional networks. These observations suggest quantitative measurements 
of topology in vertebrate microcircuits could produce similarly interesting results. 

It has been observed in local circuit preparations that a bias exists among layer 5 pyra- 
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midal neurons of rat neocortex towards strong reciprocal connectivity jl] [15], and towards 
looping motifs among triplets of this neuronal class P~5]. Furthermore, cyclic connections 
are strongest among those neurons connected by the strongest synaptic weights. These same 
neurons also exhibits STDP at the excitatory synapses that join them [5]. Given our analy- 
sis, it is now clear that these observations contradict each other; specifically, we have shown 
that standard STDP under normal spiking conditions with random uncorrelated inputs is 
loop-eliminating. Therefore, other mechanisms and constraints than those we have analyzed 
must be at play. 

Consider the case of networks that have recently spiked at abnormal rates, either due 
to increased excitability within the network (e.g., due to injury, epilepsy, etc.) or due to 
otherwise elevated extra-network inputs. If the majority of post-synaptic potentials imme- 
diately cause action potentials, standard STDP may have the effect of strengthening loops. 
Also, a network driven by highly and specifically correlated inputs may spike in temporal 
patterns conducive to loop-strengthening by STDP (a hypothesis we are currently studying). 
Finally, as we have shown, a spiking network that has recently experienced a reversal of the 
polarity of STDP will also show an increase in the number of loops observed. Clearly more 
experiments and observation would be required in order to confirm or rule out each of these 
mechanisms. 

We observe that network activity propagates smoothly through the feed-forward topology 
generated by STDP (Fig. 5 A, top panel) without segregating neurons by average spike-rates 
(Fig. 5B, upper right panel). The effect on global brain function of such properties would 
include stable average firing rates shared among all neurons, regardless of their topological 
position, and robust signal propagation, similar to "synfire chains" [TTl [18]. Finally, our 
theory holds that the reversal of STDP's polarity represents a local switch for the modi- 
fication of both global brain network topology and global brain dynamics. Thus sources 
of modulation JTH] that accomplish this reversal locally are in fact regulating global brain 
function by means of this switch. 
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IV. METHODS 



A. Simulation 

The simulation methods of Song and Abbot [8] were used to simulate each neuron in our 
100 neuron network. We observed the reported topological results in each simulation after 
10 seconds of network activity. In some cases, we performed additional, longer simulations 
of network activity to explore the convergence and stability of these topological measures 
under various conditions. Briefly, each neuron model was integrate- and-fire, with membrane 
potential determined as in [8], by r m ^- = V rest - V + g eX c(t)(E exc — V) + g in h(t)(E inh - V), 
with r m = 20ms, V rest = -60mV, E exc = OmV, E inh = -70mV, V th resh = -54mV, and 

Vreset = -60mV. 

The synaptic conductances g exc and gi nn were modified by the arrival of a presynaptic 
spike, as in [8], such that g exc {t) ->■ g exc (t) + g a , and g inh (t) -> g exc {t) + g inh . In the 
absence of a spike, these quantities decay by T ex J^^- = —g exc , and = —ginh, with 

Texc = r in = 5.0, g in = 0.015, < g a < g max , and g max = 0.01. We initialized all elements g a 
to different values for intra-network (g a = 0.005) and extra-network (g a = 0.01) inputs. 

For extra-network inputs, excitatory homogeneous Poisson spike trains were generated 
at a rate of 20 Hz. Inhibitory, inhomogeneous Poisson spike trains that model fast local 
inhibition were generated at a rate r min < r inh < r max , where r min = 5 Hz, and r max = 1000 
Hz. On each time step, dt — 0.1 ms, r in h is incremented by an amount proportional to the 
fraction, 7, of network neurons that spiked during that timestep, and decays with a time 
constant, r r , such that Tr^f^ = — [finh + (r ma x — fmin)j]- After this update, if r in h exceeds 

Tmaxi Tinh ' Tmax- 

For all simulations we report here, the STDP update rule for a synapse from neuron j 
to neuron i was g a (i,j) = g a (hj) + ga(hj)^M(i) for synaptic potentiation, and g a (i,j) = 
9a(hj) + {g m ax - 9a{h j)Y l Pa(i, j) , for synaptic depression, \i = 0.1 (g a is maintained in the 
interval [g m in, gmax}) ■ As in [8], M(i) and P(i,j) decay exponentially, such that r-^M- — 
—M{i) and T+^f = — P a , r + — r - — 20. Also as in [8], M(i) is decremented by A_ every 
time a neuron i generates an action potential, = 0.00035, and P a (i,j) is incremented 
by A + every time a synapse onto neuron % from neuron j receives an action potential, 
A + = 0.00035. This update rule effectively implements the asymmetric function of STDP 
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(see Fig. 1). 



B. Analysis 

To randomize our networks for analysis (Figs. 2B, 2C, 4C, 5), we created a random 
sequence of indices ranging uniquely from 1 to n, where n was the number of off diagonal 
elements in our network's weight matrix. We used these indices to shuffle uniquely the 
positions of all off diagonal elements in the matrix, thus preserving the network's learned 
weight matrix, while destroying its learned topology. 

We chose to sample unique loops (Fig. 2C) in the networks rather than enumerating them, 
since for long loops (k > 20) the number of possible paths to search would exceed 10 20 . We 
therefore constructed one million random paths of length k — 1 for each loop length. We 
term these paths "unique" because we sampled units within each without substitution (i.e., 
no sub-loops were sampled in which a single node is traversed more than once). However, we 
allowed that each path could be represented more than once in the one million constructed 
paths. In fact, for the shortest paths constructed (k < 3) this was necessarily the case since 
the total number of possible unique paths is less than one million. 

We sampled unique loops from adjacency matrices constructed such that the weight 
threshold produced a matrix that was precisely half-full (for 100 neurons, a matrix with 5, 000 
ones and 5, 000 zeros). For 4.0, 2.0, 1.0, 0.5, and 0.1 millisecond delays, these thresholds were 
0.0032, 0.0030, 0.0033, 0.0037, and 0.0046. In this way, we controlled across experiments 
for varying weight distributions, and sampled the same number of links for loops across 
all experiments. Counts were compared against the randomized network constructed from 
5, 000 links. 

To detect network events (Fig. 5B, 5C), we employed the method described by Thivierge 
and Cisek [20]. Briefly, for each simulation in which network events were detected, we 
generated spike trains at 1 msec resolution for each neuron, equal in duration to the time 
series analyzed and comprising the same number of spikes as observed for that neuron. We 
constructed network spike time histograms of these spike trains across all neurons using a 
bin width of 10 msec. We then determined a threshold for the network equal to a count 
which 5% of these bins exceeded. Thresholds were determined 1000 times for each network 
and each simulation, and the mean of these 1000 values used as the threshold above which 
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network events were detected in network spike time histograms for each simulation. 
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Appendix A: Supplementary Information 



1. Update rule for the weight matrix 

The classical definition of Hebbian learning for the weight w connecting two dynamical 
variables x(t) and y(t) can be written, in its simplest form, as: 

AW = rjC xy (Al) 

/oo 
x{t)y{t)dt (A2) 
-oo 

Where r\ is the learning constant, which for exposition's sake will be set to 1. It is important, 



however, to keep in mind that in order to write Eqs. |A1 A2 we are assuming an adiabatic 



approximation, i.e. the learning is small enough that the system can be considered to be in 
steady-state for the purpose of computing the correlation. 



A natural extension of Eqs. AL A2 is to introduce time, i.e. to consider delayed as well 
as instantaneous correlations: 

/oo 
C xy (t)S(t)dt (A3) 
-oo 



— oo 
oo 



C xy (t)= / x(t'-t)y(t')dt' (A4) 



It is assumed that the time-dependent weight function vanishes for long delays, 
lim^-i-oo S(t) = 0; the classical learning rule is recovered when S(t) = 5(t). If, as ex- 
perimental results describing STDP strongly suggest [5] , the weight function displays strict 
temporal anti-symmetry, i.e. S(t) = —S(—t), then 

/0 poo 
C xy (t)S(t)dt+ j C xy (t)S{t)dt (A5) 
-oo JO 

POO 

/ [C xy (t)-C xy (-t)]S(t)dt (A6) 
Jo 

A multi-dimensional linear system driven by uncorrelated input can be described as: 

±(t) = Wx(t) + f(t) (A7) 

where each unit is independently subject to Gaussian white noise £(£), a vector whose 
components satisfy (6i(0£i( s )) — cr 2 5ij5(t — s). The lagged correlator is related to the 
zero-lagged correlator by [21]: 
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C(t) 



(AS 



e w ^C t < 
C e wTt t > 

where for notational convenience we name Co = J x T (t)x(t)dt, i.e. the correlator at zero 
lag, by construction a symmetric matrix. Hence the expression for the learning update is: 



AW 



C e 



— e bo 



S(t)dt 



(A9) 



The temporal behavior of the weight function has been approximated by a piece-wise 
exponential form: 

+e*/ r t < 
S{t) = { t = (A10) 

-e~ t/T t > 

where r is STDP's time-constant, i.e. it expresses the window over which the plastic changes 
due to temporal coincidence are significant. Assuming that the network connections are only 
excitatory, and expressing without further loss of generality W = —I + A, we derive the 
synaptic weight update AW = AA as follows: 



Given that 



We obtain 



Leading finally to 



AA — — I S(t)e wt C dt + / S(t)C e w l dt 
'o Jo 



POO POO 

/ e-^e^dt = / e-^ I+wt dt = -[W- l/rl]- 1 
Jo Jo 

AA = - [W - 1/r/r 1 C + Co [W T - l/rl]' 1 



AA 



1+r 



-.4 



-i 



Co — Cq 



T 



1 + r 



-A 



-i 



(All) 
(A12) 
(A13) 

(A14) 



after dropping the multiplying constant t/(1 + r). From this expression it is possible to 
derive that the weight update is anti-symmetric, and that a perfectly symmetric system 



would not be modified, as C would commute with A (see below, Eq. A20). Of course, 
any small initial asymmetry will eventually be blown up. We can also see that STDP's 
time constant also introduces the same multiplying factor r/(l + r) for A, which can be 
absorbed by a renormalization; we will assume therefore r/(l + r) — > 1 for the remaining of 



the exposition. Consistently, the limiting behavior of Eq. A14 implies AA(r — > 0) = 
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2. Minimization of loops and dynamics 



Now we can estimate the effect of the synaptic time-dependent plasticity expressed by 



Eq. A14 on the topology of the network. For this, we will postulate a penalty or energy 
function for what we will call "loopiness" of the network. A measure of the number of loops 
occurring in the network can be obtained by summing the trace of the exponentiation of the 
network connectivity matrix, J^ fe tr [v4 fc ] /k. This loop density can be simply minimized by 
making the connections vanish, so we need to introduce a regularization penalty to avoid 
this effect; an obvious measure of the strength of the connections in a network is tr |/4^4 T ] , 
which in a binary graph would be equivalent to the total number of links. We postulate 
then the following "loopiness" energy: 

k 

The change in this energy upon small changes AA is expressed as AS ~ tr [<9a£At4 t ] ; 
it can be easily verified that d^S — ( I — A T ^j 1 — A, and therefore: 

AS tr [Ki] - tr [K 2 ] 

K x = {I-A T Y l [(i-Ar'Co-Coil-A 7 )- 1 } 

K 2 = A [C (I - A?)- 1 -(I- A)- l C ] (A16) 

We will demonstrate in what follows that the traces of K\ and K 2 are strictly semi-positive 



under the synaptic changes elicited by STDP (i.e. Eq. A14), and therefore the loopiness 
energy can only decrease over time. We will consider firs that the correlation matrix can be 
approximated by the identity, i.e. Cq ~ /. This case can be solved analytically; we will show 
numerically that the same conclusions hold for the generic case of any Cq that results from 
the dynamical system determined by A, drawn from a Gaussian or Poisson distribution. 
Let us consider tr [Ki], rewritten as: 

tr [(/- A T Y\I- A)- 1 - (I- A T Y 2 } (A17) 

and which is of the from 

tr [(PP T -P 2 )] (A18) 

For any matrix P, 

tr [(P T - P)(P T - P) T ] = tr [RR T ] > 
16 



which upon expanding leads to 



tr [PP T - P 2 ] > 



(A19) 



and ensures the positivity of tr [Ki] , under the assumption Co ~ /. Moreover, we can show 
that this is valid for an arbitrary Co- Under the assumption of stability and homogeneous 
Gaussian noise, the correlation and weight matrices are related by the Lyapunov equation 

[2HE2]: 



WCn + C W 



-QQ 1 



(A20) 



where QQ T is the generalized temperature tensor of the noise, whose components QiQj = 
afj are the corresponding noise variances. For the case we are considering, QQ T = I, a 
symmetric system W = W T has the solution Co = — W~ 1 /2. A formal solution for the 
general case is [23] 

POO 

C = / e wTt e wt dt (A21) 
Jo 

Assuming homogeneous noise, this reduces in our case to 



Co 



e -2It e ATt e M dt 



(A22) 



Following the derivation in A18 A19, the full expression for tr [K\] can be written as: 



tr [K x \ = tr [RR T C ] 



(A23) 



Given that 



tr 



RR T e AT e A 



tr [{e A R){e A R) T ] >0 



It follows through [A22] that tr [Id] > 
NEW STARTS HERE @@@@@@@@ 



Similarly, the second term in Eq. A16, tr [K?], can be rewritten as 



tr [A T {I-A)- l C Q ] -tr [A{I-AY l C Q ] 



which can be reduced to 



tr [(A T - A){I - AY l C Q ] 
Replacing A(I — A)' 1 by (J — A)^ 1 — I, the term can be transformed to 



(A24) 



tr [C ] - tr [(J - A T )(I - A)- 1 C ] = tr [C ] - tr [W T W- l C ] 



(A25) 
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Assuming that QQ T = I in Eq. A20 and pre-multip lying by W 1 , we obtain 



from which we derive: 



w- x c Q w T = -W~ l - C 

tr $V T W- l C Q ] = -tr [W- 1 ] - tr [C ] 
tr [AT 2 ] = 2tr [C ] + tr [W' 1 } 



(A26) 
(A27) 

(A28) 



Now we can use the formal solution of the Lyapunov equation, Eq. A21, and the fact that 



for a stable matrix such as W (i.e. all its eigenvalues have negative real components) we 
can write 



W' 1 



dt 



and modify Eq. A28 accordingly: 



tr [K 2 ] = 2 tr [e wt e wTt ]dt - / tr [e wt ]dt 
Jo Jo 



(A29) 



Given that for any matrix, the eigenvalues satisfy \(e w ) = e xlyW \ we get 

,00 N roo N 



tr 



poo J * POO iy 

[K 2 ] = 2 / T e2MXh)tdt - / E' 
Jo k=i Jo k=i 



dt 



(A30) 



where are the eigenvalues of W. Calling = — fik + ijki > 0, the first term in the 
r.h.s. above is 



Jo k=1 Jo k=1 



Now we can compare both terms in the r.h.s. of Eq. A30 for each k and t: e Mftt > e 



-li k t+i~ikt 



which leads directly to tr [if 2 ] > and completes the proof of the semi-negativity of the 



changes in the energy function (Eq. A16). 

Interestingly, this result is related to a property of M-matrices. For an M-matrix: (1) 
the off-diagonal elements are semi-negative, Mjj < Vz 7^ j, and (2) is positive stable, 
Re[Aj(M)] > VI It can be shown that for any M-matrix of dimension A^ (Th. 5.7.23 in 
|23j ) tr [M T M _1 ] < tr [I] = N. Choosing — W as an M-matrix, the theorem leads to a 
similar result for the non- negativity of tr[Ay when Co is close to the identity. 

We have assumed throughout that the system is in a regime of dynamical stability, and 
presented a case for the stabilizing effect of STDP by linking loops and eigenvalues in Eq. |3j 
It follows that loopiness minimization (Eq. [2j is equivalent to maximization of stability (as 
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defined by the l.h.s. of Eq. |3J), constrained by the total matrix weight. We can further 
understand this by explicitly expanding to first order the update equation A14| to see the 
effect on U = — J^log \K\- Assuming again that QQ T = I, the solution to the Lyapunov 
equation (Eq. A21) can be approximated by a series expansion in powers of A |23j; to first 
approximation Co — / + \{A + A T ), leading to AA ~ A — A T . Through Eq. [3] we obtain 
SLA ~ |tr [A 5 A + 5 A A], and in turn SLA ~ (tr [A 2 ] - tr [AA T ]) < 0, making the system 
more stable. 

UP TO HERE @@@@ 
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FIG. 1: Schematic of the topological effect of STDP. Feedback connections in an initial topology 
(left) from first (1) and second (2) order "follower" neurons (light blue) to a "trigger" neuron 
(green) create loops of length k = 2 and k = 3. These connections are selectively penalized by 
the STDP learning rule (lower middle, red). The plot (middle) depicts this rule, with the time 
difference between follower (black) and trigger (green) action potentials on the a>axis, and the 
expected synaptic modification on the y-axis. When spikes successfully propagate through the 
loopy network they feed back to the trigger, arriving at the follower-trigger synapse immediately 
after the trigger neuron fired and resulting in synaptic depression (red). Through repeated spike 
propagation events, STDP results in a completely feed forward learned topology (right) to the 
output neuron (dark blue). 
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FIG. 2: Global topological effects of STDP. A) A monotonic decrease in the loopiness measure (see 
Equation [2j first term) over time is observed in a simulated network of 100 neurons undergoing 
STDP. Simultaneously, STDP results in a net increase in the weightedness of the network (inset, 
see Equation [2| second term) . Shown here and in B) is the average of 8 separate simulations of 20 
seconds of network activity; error bars are standard deviation. B) Number of closed loops of length 
5, 3, and 2, decreases as a function of weight threshold for network connections. Dotted lines show 
counts for randomized networks with same number of total connections. C) Number of unique loops 
sampled from five networks with varying synaptic delays following 10 seconds of simulated activity, 
and from a random network with 5, 000 connections. Number of loops is shown as a function of 
loop length. Loops were sampled across different learned networks while maintaining the number 
of network connections at 5, 000 by varying the weight threshold (from 0.003 to 0.0046) for each 
delay. Greater synaptic delays decreases the loop-eliminating topological effect of standard STDP. 
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FIG. 3: Local topological effects of STDP. A) Inverse relationship of in-degree versus out-degree of 
intra-network connections for each neuron in a network after 10 seconds of STDP across multiple 
weight thresholds for network connections. Colors in left and right panels correspond to a weight 
threshold used to construct the network over which degrees were measured. The color key can 
be read from the left panels' vertical axes and corresponding color found along each manifold. 
B) Correlated extra-network in-degree and intra-network out-degree indicate an opposite effect of 
STDP on extra-network inputs. 
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FIG. 4: Effect of reverse STDP. A) In-degree versus out-degree of intra-network connections fol- 
lowing different durations and polarities of STDP, shows a strong inverse relationship for standard 
STDP. Adjacency weight threshold was 0.005. Green markers correspond to the network after 1 
second of standard STDP, followed by 3 or 5 seconds of standard STDP (blue markers) or reverse 
STDP (red markers). B) Total synaptic input versus output for intra-network connections shows a 
similar inverse relationship for standard STDP. C) Number of closed loops of length 5, 3, and 2, is 
decreased by standard STDP, and restored with reverse STDP (plotted as in Fig. 2B). D) In-degree 
of extra-network inputs versus out-degree of intra-network outputs, plotted as in A, shows a strong 
positive correlation for standard STDP. Adjacency weight threshold was 0.007 for extra-network 
inputs and 0.005 for intra-network outputs. E) Total synaptic extra-network input versus total 
synaptic intra-network output, plotted as in B, shows a similar positive correlation. In each panel 
the topological effects of STDP are reversed by^ajeverse STDP. 




FIG. 5: Dynamical effects of STDP. A) Raster plot of the spiking activity for a network after 
STDP (top), and for a surrogate network where intra- network weights were reassigned randomly 
to network connections, thus destroying STDP-learned topology (bottom). Each point corresponds 
to a spike for each neuron. Each neuron was assigned a rank according to the sum of its extra- 
network input weights, with the lowest rank corresponding to the highest sum. B) Peri-event time 
histograms for each neuron in the STDP network (top left) and its surrogate (bottom left), pooled 
across 8 separate simulations (bin width, 2msec). Histograms show different network propaga- 
tion properties. Spike counts and extra- network weights for the same networks do not co-vary 
in the STDP-learned topology (top right), but are highly correlated for the surrogate (bottom 
right). C) Peri-stimulus time histograms summed across all simulations and all neurons for the 
STDP network (blue) and three surrogates, in which the intra-network connections (red), extra- 
network connections (green) or both (magenta^gwere randomized (top). Skewness versus kurtosis 
of these histograms averaged across 8 separate simulations each (bottom, error bars show stan- 



